System and method for casting content

ABSTRACT

A system and method is provided that provides for the transfer of the execution of content from a user device to an external device for output of the content by the external device. External devices may be detected in a physical space, and identified based on previous connection with the user device, based on a shared network or shared system of connected devices including the user device, based on image information captured by the user device and previously stored anchoring information that identifies the external devices, and the like. An external device may be selected for potential output of the content based on previously stored configuration information associated with the external device including, for example, output capabilities associated with the external device. The identified external device may output the transferred content in response to a user verification input, verifying that the content is to be output by the external device.

CROSS REFERENCE TO RELATED APPLICATION

This application is a 35 U.S.C. § 371 National Phase Entry Applicationfrom PCT/US2020/070793, filed Nov. 16, 2020, designating the U.S., thedisclosure of which is incorporated herein by reference in its entirety.

BACKGROUND

Connected, or networked, or compatible devices allow users to consume,share and control content using different devices. Some systems rely ona series of user inputs to scan for and detect compatible devices,initiate sharing and/or mirroring of content, control compatibledevices, and the like. This can result in a complicated and cumbersomeuser experience, and may discourage users from taking advantage of thiscapability.

SUMMARY

In one general aspect, a computer-implemented method includes executing,by a processor of a user device, content on the user device; detecting,by the user device, at least one external device within a set range ofthe user device; identifying the at least one external device for outputof the content being executed by the user device based on outputcapabilities of the at least one external device and outputcharacteristics of the content being executed by the user device; andtransferring execution of the content from the user device to the atleast one external device for output by the at least one externaldevice.

In some implementations, detecting the at least one external devicewithin the set range of the user device includes detecting the at leastone external device within a field of view of the user device. In someimplementations, identifying the at least one external device includesdetecting physical features in image information captured by the userdevice; identifying a location of the user device based on the detectedphysical features; and identifying the at least one external devicebased on the identified location of the user device. In someimplementations, the user device includes a handheld device and a headmounted display device, such that executing content includes executingcontent on the handheld device; and detecting physical features in theimage information includes detecting physical features in imageinformation captured within the field of view of the head mounteddisplay device. In some implementations, identifying the location of theuser device based on the detected physical features includes identifyingthe location of the user device based on a comparison of previouslystored features in a database and accessible to the user device via aserver.

In some implementations, detecting the at least one external devicewithin the set range of the user device includes at least one ofdetecting the at least one external device as a previously connecteddevice; detecting the at least one external device based on anelectromagnetic signal; detecting the at least one external devicewithin a shared network; or detecting the at least one external devicewithin a network of previously connected devices. In someimplementations, detecting the at least one external device within theset range of the user device includes detecting the at least oneexternal device within a field of view of a user of the user device.

In some implementations, transferring the execution of the content fromthe user device to the at least one external device for output by the atleast one external device includes outputting a request for userverification prior to transferring the content from the user device tothe at least one external device; receiving a user verification inputverifying the transfer of content from the user device to the at leastone external device; and transferring the execution of the content fromthe user device to the at least one external device in response toreceiving the user verification input. In some implementations,outputting the request for user verification includes outputting atleast one of a visual request or an audible request, and receiving theuser verification input includes receiving at least one of a touch inputor an audible input.

In another general aspect, a system includes an output system includingan audio output device and an image output device; a sensing system; atleast one processor; and a memory. The memory stores instructions that,when executed by the at least one processor, cause the at least oneprocessor to output, by the output system, content executed by the atleast one processor; detect, by the sensing system, at least oneexternal device within a set range of the sensing system; identify theat least one external device for output of the content being executed bythe user device based on output capabilities of the at least oneexternal device and output characteristics of the content being outputby the output system; and transfer execution of the content to the atleast one external device for output by the at least one externaldevice.

In some implementations, in detecting the at least one external devicethe instructions cause the at least one processor to at least one ofdetect the at least one external device based on image informationcaptured by the sensing system; detect the at least one external deviceas a previously connected device; detect the at least one externaldevice based on an electromagnetic signal; detect the at least oneexternal device within a shared network; or detect the at least oneexternal device within a network of previously connected devices. Insome implementations, in detecting the at least one external devicebased on image information captured by the sensing system, theinstruction cause the at least one processor to detect physical featuresin the image information; identify a location based on a comparison thedetected physical features and with previously stored features in adatabase accessible to the system via a server; and identify the atleast one external device based on the identified location.

In some implementations, in transferring the execution of the content tothe at least one external device, the instructions cause the at leastone processor to output a request for user verification prior totransferring the content to the at least one external device; receive auser verification input verifying the transfer of content to the atleast one external device; and transfer the execution of the content tothe at least one external device in response to receiving the userverification input. In some implementations, the request for userverification includes at least one of a visual request or an audiblerequest, and the user verification input includes at least one of atouch input or an audible input. In some implementations, the outputsystem, the sensing system, the at least one processor and the memoryare installed in at least one of a handheld electronic device or a headmounted display device that are operable within a network.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram of an example system, in accordance withimplementations described herein.

FIGS. 2A-2E are front views of example user electronic devices in theexample system shown in FIG. 1.

FIG. 3 is a block diagram of an example system configured to implementthe concepts described herein.

FIGS. 4A-4F illustrate detection of external devices and transfer ofexecution of content from a user device to the identified externaldevice, in accordance with implementations described herein.

FIG. 5 is a flowchart of an example method, in accordance withimplementations described herein.

FIG. 6 shows an example of a generic computer device and a genericmobile computer device.

DETAILED DESCRIPTION

Users may consume content using a number of different types of devices,including, for example, handheld devices (smartphones and the like),wearable devices such as hand and/or wrist worn devices (smart watches,smart bracelets, smart rings and the like), head mounted devices (smartglasses, goggles, headsets and the like), ear worn devices, neck wornlanyard devices, other mobile devices (tablet computing devices, laptopcomputing devices and the like), desktop computing devices, smarttelevisions, smart speakers, and other such devices. A system andmethod, in accordance with implementations described herein, providesfor the sharing and transferring of the execution of content, executedand output by a user device, from the user device to a detected externaldevice, for execution and output by the external device, in response tothe detection of the external device. For example, the execution ofcontent may be transferred or shared for output by the external devicein response to the detection of the external device within a setproximity or a set range of the user device, detection of the externaldevice within a field of view of the user device, detection of theexternal device based on image data collected by the user device,detection of the external device within a shared local network,detection of a set signal strength associated with the external device,and other such detection conditions. In some implementations, thedetection of the external device for sharing or transfer of the outputof content is accomplished without user intervention. In someimplementations, content is shared or transferred for output by thedetected external device without user intervention. In someimplementations, content is shared or transferred for output by thedetected external device in response to a user input indicatingverification or confirmation. In some implementations, the detection ofthe external device and/or the transfer of content from the user deviceto the external device for execution by the external device may becarried out with limited user intervention to enhance the userexperience in consuming content within a system of connected devices.

In some situations, the user experience may be improved by sharing ortransferring the execution of content to a detected external device, foroutput by the detected external device. For example, a user viewingcontent, for example video content on a handheld device such as asmartphone may prefer to view the content on an alternative outputdevice, such as, for example, a television display screen, a computerdisplay screen and the like having a larger image output capability whenthe larger image output device is available. Similarly, a user listeningto music through, for example, ear buds, an audio output device of ahead mounted wearable and the like, may prefer to listen to the musicthrough an alternative output device such as a smart speaker, a soundsystem and the like when the alternative audio output device isavailable. The user experience may be further improved if theidentification and transfer of content for output by the alternativeoutput device can be facilitated by the detected previous connection ofdevices, detection of devices within a shared network or a system ofpreviously connected devices, image recognition based on previouslystored anchoring information associated with the alternative outputdevices and the like, which may reduce the amount of user interactionand input required to complete the transfer for execution of the contentby the alternative output device. Hence, the term output capabilitiescan be used to refer to a quality with which the external device canperform a task or a set of tasks, and/or can be used to refer to theoutput of image (still and/or moving images) and/or audio output thatdistinguish and/or uniquely define the external devices. The term outputcharacteristics of the content can be used to refer to a certain type ofoutput data (for example, image data and/or audio data) and/or amount ofdata associated with the output.

FIG. 1 illustrates a user in connection with an example system 100, inaccordance with implementations described herein. In the example shownin FIG. 1, the user is wearing a wearable device 110 in the form of anexample head mounted display (HMD) device 110, or smart glasses 110, awearable device 130 in the form of an example wrist worn device 130, ora smartwatch 130, and a wearable device 140, in the form of an exampleear worn device 140, or ear bud(s) 140, simply for purposes ofdiscussion and illustration. In the example shown in FIG. 1, the user isholding a handheld device 120 such as, for example, a smartphone 120,simply for purposes of discussion and illustration.

The example system 100 may include one or more computing and/orelectronic devices that can communicate with each other, for example,over a network 195 and/or over alternative network(s) and/directly witheach other. Example client devices, or user devices, may include, forexample, the example devices 110, 120, 130, 140 worn and/or operated bythe user, as well as other electronic devices including, for example, adisplay screen 150/television monitor 150, an audio output device155/smart speaker 155, a laptop or notebook computing device 160, atablet computing device 170, a desktop computing device 180, and othersuch devices. One or more external resources 190 may be accessible tothe devices via the network 195. The external resources 190 may include,for example, a server providing access to one or more databases and thelike.

FIG. 2A is a front view of the example wearable device 110, or HMD 110,worn by the user in FIG. 1. FIG. 2B is a front view of the examplehandheld device 120, or smartphone 120, held by the user in FIG. 1. FIG.2C is a front view of the example wearable device 130, or smartwatch130, worn by the user in FIG. 1. FIG. 2D is a perspective view of theexample wearable device 140, or earbuds 140, worn by the user in FIG. 1.FIG. 2E is a perspective view of another example head mounted wearabledevice 110′.

The wearable device 110, in the form of the HMD 110, or smart glasses110 in the example shown in FIG. 2A, may include a frame 111, with adisplay device 112 coupled in the frame 111. In some implementations, anaudio output device 113 is coupled to the frame 111. In someimplementations, a touch surface 114 allows for user control, input andthe like of the HMD 110. The HMD 110 may include a sensing system 116including various sensing system devices and a control system 117including various control system devices to facilitate operation of theHMD 110. The control system 117 may include a processor 119 operablycoupled to the components of the control system 117 and a communicationmodule 115 providing for communication with external devices and/ornetworks. The HMD 110 may also include an image sensor 118 (i.e., acamera 118), a depth sensor, a light sensor, and other such sensingdevices. In some implementations, the image sensor 118, or camera 118 iscapable of capturing still and/or moving images, patterns, features,light and the like.

Similarly, the wearable device 110′ illustrated in FIG. 2E is shownattached to a pair of glasses to be worn by the user, for use as a headmounted wearable device 110′. However, the wearable device 110′ may beattached to, for example, ear buds, lanyards, bracelets, and other suchdevices that can be worn by the user. In some implementations, thewearable device 110′ includes a frame 111′, an audio input and/or outputdevice 113′ (i.e., a microphone and/or a speaker) and an image sensor118′ (i.e., a camera 118′), a depth sensor, a light sensor, and othersuch sensing devices). In some implementations, a touch input surface114′ allows for user control, input and the like of the wearable device110′. The wearable device 110′ may include a control system 117′ and aprocessor 119′ to facilitate operation of the wearable device 110′, anda communication module 115′ providing for communication with externaldevices and/or networks.

The wearable device 130, in the form of the smartwatch 130 in theexample shown in FIG. 2B, may include an interface device 131. In someimplementations, the interface device 131 functions as an output device,including, for example, a display area 132 that can output informationto the user. In some implementations, the interface device 131 functionsas an input device, including, for example, a touch surface 133,allowing the interface device 131 to receive touch inputs from the user.In some implementations, the interface device 131 can function as aninput device and an output device. The wearable device 130 may include asensing system 136 including various sensing system devices. Thewearable device 130 may include a control system 137 including variouscontrol system devices, a communication module 135 providing forcommunication with external devices and/or a network, and a processor139, to facilitate operation of the device 130. The wearable device 130may also include an image sensor 138 (i.e., a camera 138), a depthsensor, a light sensor, and other such sensing devices. In someimplementations, the image sensor 138, or camera 138 is capable ofcapturing still and/or moving images, patterns, features, light and thelike.

The handheld device 120, in the form of the smartphone 120 in theexample shown in FIG. 2B, may include an interface device 121. In someimplementations, the interface device 121 functions as an output device,including, for example, a display area 122 that can output informationto the user. In some implementations, the interface device 121 functionsas an input device, including, for example, a touch surface 123,allowing the interface device 121 to receive touch inputs from the user.In some implementations, the interface device 121 functions as an inputdevice and an output device. The handheld device 120 may include asensing system 126 including various sensing system devices. Thehandheld device 120 may include a control system 127 including variouscontrol system devices, a communication module 125 providing forcommunication with external devices and/or a network, and a processor129, to facilitate operation of the handheld device 120. The handhelddevice 120 may also include an image sensor 128 (i.e., a camera 128), adepth sensor, a light sensor, and other such sensing devices. In someimplementations, the image sensor 128, or camera 128 is capable ofcapturing still and/or moving images, patterns, features, light and thelike.

The wearable device 140, in the form of the earbuds 140, is shown inFIG. 2D. The wearable device 140 may include an audio device 143. Insome implementations, the audio device 143 functions as an audio outputdevice (i.e., a speaker) and an audio input device (i.e., a microphone).In some implementations, a touch input surface 144 allows for usercontrol, input and the like of the wearable device 140. The wearabledevice 140 may include a control system 147 and a processor 149 tofacilitate operation of the wearable device 140, and a communicationmodule 145 providing for communication with external devices and/ornetworks.

FIG. 3 is a block diagram of an example system 300 that can implementthe concepts described herein. The block diagram shown in FIG. 3illustrates a first electronic device 310, such as, for example, one ofthe example user devices 110, 120, 130, 140 shown in FIG. 1, incommunication with a second electronic device 315 such as, for example,one of the example external devices 150, 160, 170, 180, 190 shown inFIG. 1.

The first electronic device 310 may include a sensing system 360 and acontrol system 370. The sensing system 360 may include one or moredifferent types of sensors, including, for example, a light sensor, anaudio sensor, an image sensor, a distance/proximity sensor, and/or othersensors and/or different combination(s) of sensors. In someimplementations, the sensing system 360 includes an image sensor and/oran optical sensor. The control system 370 may include, for example,power/pause control device(s), audio and video control device(s),optical control device(s), and/or other such devices and/or differentcombination(s) of devices. The sensing system 360 and/or the controlsystem 370 may include more, or fewer, devices, depending on aparticular implementation. The electronic device 310 may include aprocessor 390 in communication with the sensing system 360 and thecontrol system 370. The processor 390 may process inputs received fromthe sensing system 360, such as, for example, images and/or opticalscans, captured by the image sensor/optical sensor. The first electronicdevice 310 may include an input system 340 that can receive user inputsto be processed by the processor 390 and output by an output system 350under the control of the control system 370. The input system 340 mayinclude various types of input devices including, for example, a touchinput surface, audio input devices that can receive audio inputs(including, for example, audio sensors, or microphones, included in thesensing system 360), a gesture recognition device (including, forexample, images captured by image sensors(s) of the sensing system 360and processed by the processor 390), and other such input devices. Theoutput system 350 may include various types of output devices such as,for example, display device(s), audio output device(s), or speakers,physical and/or tactile output devices, and other such output devices.The first electronic device 310 may include a memory 380, and acommunication module 330 providing for communication between the firstelectronic device 310 and one or more other, external device(s),networks, servers and the like, including, for example, the secondelectronic device 315.

The second electronic device 315 may include a sensing system 365including one or more different types of sensors, and a control system375 control modules for controlling operation of the second electronicdevice 315. A number and/or combination of the components of the sensingsystem 365 and/or the control system 375 may vary, depending on aparticular implementation. The second electronic device 315 may includea processor 395 in communication with the sensing system 365 and thecontrol system 375, allowing the processor 395 to process inputsreceived from the sensing system 360. The second electronic device 315may include an input system 345 that can receive inputs to be processedby the processor 395 and output by an output system 355 under thecontrol of the control system 375. The output system 355 may includevarious types of output devices such as, for example, display device(s),audio output device(s), or speakers, physical and/or tactile outputdevices, and other such output devices. In some implementations, thesecond electronic device 315 includes a memory 385, and a communicationmodule 335 providing for communication between the second electronicdevice 315 and one or more other, external device(s), networks, serversand the like, including, for example, the first electronic device 310.

FIGS. 4A-4F illustrate the detection and/or identification of externaldevices and the transfer of the execution of content, or casting ofcontent, from a user device to the identified external device, inaccordance with implementations described herein. In the example to bedescribed with respect to FIGS. 4A-4F, the user device refers to thedevice, or devices, in use by the user for accessing content, and fortransferring execution of content, or casting content to the externaldevice, and the external device refers to the device, or devices, in thephysical space that are configured to receive and output content fromthe user device(s). In the example to be described with respect to FIGS.4A-4F, the handheld device 120 and the wearable device 110 (in the formof smart glasses 110) will be used as example user devices, and thetelevision 420 will be used as an example external device that receivescontent from the user device, simply for purposes of discussion andillustration. The principles to be described may be applied to otherdevices and/or combinations of devices functioning as user devices, andother devices and/or combinations of devices functioning as externaldevices.

FIG. 4A is a third person view of a user approaching an example physicalspace 400. In the example illustrated in FIG. 4A, the user is wearingthe example wearable device 110 (i.e., the HMD 110/smart glasses 110)and the example wearable device 130 (i.e., the smart watch 130). In theexample shown in FIG. 4A, the user is accessing content 410 using thehandheld device 120. In the example scenario illustrated in FIG. 4A, thecontent 410 would be output in the display area 122 of the handhelddevice 120; however, in FIG. 4A the content 410 is illustrated in a paneoutside of the handheld device 120, simply for ease of discussion andillustration. In this example, the content 410 is image content, orvideo content. The principles to be described herein may be applied toother types of content and/or to other types of user devices thatprovide the user with access to content.

As the user moves further into the physical space 400, as shown in FIG.4B, external devices within the physical space 400 may be detected. Forexample, external devices that are capable of communication with theuser device(s) in use by the user (for example, the handheld device 120and/or the wearable device(s) 110, 130) may be detected. Detectionand/or identification of the external device(s) may provide for theexchange of information between the user device(s) and the detectedexternal device(s), allowing the user to transfer, or share, or castcontent to the detected external device(s) for output by the externaldevices, to provide an improved user experience.

In some implementations, the external device(s) are detected based on apreviously established connection or pairing between the user device andthe detected external device(s). In some implementations, the externaldevice(s) are detected based on a detected signal, for example, a shortrange signal such as wife signal or a bluetooth signal, emitted by theexternal device(s), electromagnetic signals, and the like. In someimplementations, the external device(s) are detected based on a detectedsignal strength. In some implementations, the external device(s) aredetected in response to a determination that the user device(s) and theexternal device(s) are in the same local network. In someimplementations, the user device(s) (for example the handheld device 120and/or the wearable device 110) and the external device(s) includeultra-wideband (UWB) chips that provide for detection of andcommunication between the user device(s) and the external device(s) forexchange of information. In some implementations, external devices,features and/or elements in the physical space may be detected in imageinformation captured by the user device(s). Image information capturedby the user device(s) may be compared to previously stored anchoringinformation to identify features defining the physical space 400. Thismay be used to identify the physical space 400, and to identify knownexternal device(s) in the identified physical space 400, and/or todetect, locate and identify the external device(s) in the physical space400.

Returning to FIG. 4B, the example physical space 400 includes atelevision 420 mounted at an intersection of two physical boundaries, orwalls, of the physical space 400 defining a corner 490. A display of acomputing device 440 is positioned on a desk 450 at a first side of achair 460. A lamp 480 is positioned at a first side of a couch 470. Anaudio output device 430, or smart speaker 430, is positioned on a table,between respective second sides of the chair 460 and the couch 470. Thefeatures and devices in the example physical space 400 illustrated inFIGS. 4A and 4B are provided simply for purposes of discussion andillustration. The principles described herein can be carried out insituations in which more or fewer features, elements, devices and thelike are available and/or arranged differently.

As noted above, as the user moves into the physical space 400, variousexternal devices such as, for example, the television 420 and the smartspeaker 430, may be detected by the user device (the handheld device 120and/or the HMD 110 in this example). For example, the external devices(in the form of the television 420 and the smart speaker 430 in thisexample) may be detected based on a previous connection or pairing withthe user device(s) (in the form of the handheld device 120 and the HMD110 in this example), detection of the user device(s) and the externaldevice(s) in the same local network (for example, a home network orlocal area network), signals emitted by the external device(s) anddetected by the user device(s), and the like. In some implementations,the detected external device(s) are identified with associatedfunctionality. For example, the television 420 may be identified ascapable of outputting image content (still and/or moving image content,together with audio content). The smart speaker 430 may be identified ascapable of outputting audio content. A connection may be establishedbetween the user device(s) and one (or more) of the identified externaldevice(s) to facilitate the transfer of content from the user device(s)to the identified external device(s) for execution by the identifiedexternal device(s).

In some situations, as the user enters the physical space 400, variousfeatures, elements, devices and the like are captured within the fieldof view V of the user device, in the form of the HMD 110 in the exampleshown in FIG. 4C. In some implementations, image information captured bythe user device is used to detect and identify external devices in thephysical space 400. For example, the image information captured by theuser device may be used to identify external devices in the physicalspace that are compatible for the transfer, or sharing, or casting ofcontent from the user device and/or that are capable of outputtingcontent transferred to the external device from the user device.

The example physical space 400 includes numerous examples ofsubstantially consistent features which may be detected in imagescaptured by an image sensor of the user device, and which may be used toidentify the physical space 400, and to identify external devices in thephysical space 400. For example, the television 420 may be identifiedbased on the known placement of the television 420 in the living room(the physical space 400 in this example) of the user, establishedduring, for example, a setup and configuration process for devices inthe home of the user. In some implementations, the television 420 may beidentified based on detection of the corner 490 of the physical space400, and the positioning above the chair 460 and the smart speaker 430.These physical features associated with the area surrounding thetelevision 420 may substantially consistent physical features defininganchoring information associated with the television 420 in the physicalspace, and which may facilitate the later identification of thetelevision 420 in the physical space 400 in this manner. FIG. 4Dillustrates a view of the television 420, through the HMD 110, includingthese substantially consistent features. Similarly, the smart speaker430 may be identified based on the known placement of the smart speaker430 between the couch 470 and the chair 460, and proximate the corner490. This type of anchoring information may be collected and stored, forexample, in a database that is accessible via a server through a networkas described above. The user may access the stored anchoring informationfor identification of the physical space 400, and for identification ofexternal devices in the physical space 400. The user may transfer, orshare, or cast content to an external device detected and identified inthis manner, for execution by the external device.

In the example shown in FIGS. 4A-4F, the user is consuming content, forexample, video content output by the handheld device 120. As the userenters the physical space 400, one or more external devices in thephysical space 400 that are compatible for receiving the content 410from the handheld device 120 and outputting the content 410 may bedetected and identified in the manner described above with respect toFIG. 4B and/or FIG. 4C. That is, one or more external device(s) that areequipped to output the content 410 may be detected. In response to thedetection and identification of external devices in the physical space400, the system may determine that the television 420 (the externaldevice in this example) may provide for improved viewing of the content410 currently output by the handheld device 120 (the user device in thisexample). The system may make this determination based on, for example,previous connection and pairing, previous configuration of the deviceswithin the network, anchoring information including previously storedinput/output capabilities or characteristics of the television 420(compared to output characteristics of the content 410), previouslystored processing capabilities or characteristics of the television 420and the like.

In this example, the content 410 output by the handheld device 120 maybe video content 410 including image content and audio content. Thus,the television 420 may be identified as the external device in thephysical space 400 to which the video content 410 from the handhelddevice 120 is likely to be transferred, or shared, or cast for outputbased on characteristics associated with the content 410. In someimplementations, the system may generate a query 425, or prompt 425, orrequest for verification 425, as shown in FIG. 4E. The request forverification 425 may request user confirm that the content 410 should betransferred to, or shared with, or cast to the television 420 prior tocasting, to avoid unintentional or inadvertent casting of content to anexternal device. In the example shown in FIG. 4E, the request forverification 425 is a visual output by the television 420, for purposesof discussion and illustration. In some implementations, the request forverification 425 is output in a different form. For example, the requestfor verification 425 can be an audio output, an audio output combinedwith a video output, a physical output, and the like. In response to therequest for verification 425, the user may enter a verification input.The verification input may include, for example, an input at the HMD110, an input at the handheld device 120, and the like. For example, theverification input may include a touch or tap input on the touch surface114 of the HMD 110, the touch surface of the handheld device 120, orother such user input. The verification input may include an audible, orvoice input, that is detected by an audio input device, or microphone, fthe external device. In response to the user verification input, thecontent 410 being output by the handheld device 120 may be output by thetelevision 420, as shown in FIG. 4F.

In some implementations, content from the user device (the handhelddevice 120 in this example) may be transferred for execution, or castto, the external device (the television 420 in this example) withoutuser intervention. That is, in some implementations, content from theuser device (the handheld device 120 in this example) may be transferredto, or shared with, or cast to, the external device (the television 420in this example) for execution by the external device without therequest for verification 425 and the user verification input describedabove with respect to FIG. 4E.

FIG. 5 is a flowchart of an example method 500 for detecting externaldevice(s), and for transferring the execution and output of content todetected external device(s), in accordance with implementationsdescribed herein. Content may be executed and output by a first device(block 510), such as, for example, the content 410 output by the userdevice in the form of the handheld device 120 described above withrespect to FIGS. 4A-4F. One or more external device(s) may be detected(block 520, 530). The detected external device(s) may be capable ofcommunication with the first device, for example through a network, orother type of connection. In some implementations, the externaldevice(s) are detected based on previous connection to or pairing withthe first device, a shared network with the first device, a sharedsignal and/or signal strength, image information extracted from imagescaptured by the first device and/or by another user device and matchedwith previously stored anchoring information, and the like. The systemmay determine that one or more of the detected external device(s) isconfigured for output of the content currently being output by the firstdevice (block 540), and may select one of the detected external devicesfor output of the content.

For example, as described above with respect to FIGS. 4A-4F, the systemmay determine which of the detected external devices detected in thephysical space 400 are capable of outputting the content 410 currentlybeing output by the handheld device 120. In the example described above,the content 410 being output by the handheld device 120 includes atleast image content, and may also include audio content. Thus, thesystem may determine that the television 420 is the most properlyconfigured (of the external devices detected in the physical space 400)for the output of the video content 410. For example, the audio outputdevice 430, or smart speaker 430 may have limited, or no image outputcapability. Similarly, the computing device 440 may have limited, or noaudio output capability. Thus, the system may select the television 420as a second device, for output of the content 410.

In some implementations, the system outputs a request for verification,prior to transferring the execution of content from the first device tothe second device (block 550) to, for example, avoid unintentionaltransfer, or sharing, or casting of content to an external device. Inresponse to a user input verifying the transfer of content from thefirst device to the second device (block 560), the content may betransferred to the second device, for output by the second device (block570).

A system and method, in accordance with implementations describedherein, provides for the sharing and transfer of execution of contentoutput by a user device, from the user device to an external device, forexecution and output by the external device. Execution of content may betransferred or shared for output by the external device in response tothe detection of the external device within a set proximity, or a setrange, of the user device, detection of the external device within afield of view of the user device, detection of the external device basedon image data collected by the user device, detection of the externaldevice within a shared local network, detection of a set signal strengthassociated with the external device, electromagnetic signals, and othersuch detection conditions. The detection of the external device and/orthe transfer of content from the user device to the external device forexecution by the external device may be carried out with limited userintervention to enhance the user experience in consuming content withina system of connected devices.

FIG. 6 shows an example of a generic computer device 600 and genericmobile computer devices 650, 690, which may be used with the techniquesdescribed herein. Computing device 600 is intended to represent variousforms of digital computers, such as laptops, desktops, tablets,workstations, personal digital assistants, televisions, servers, bladeservers, mainframes, and other appropriate computing devices. Forexample, computing device 600 may be and/or be used as the serverreferenced above. Computing device 650 is intended to represent variousforms of mobile devices, such as personal digital assistants, cellulartelephones, smart phones, and other similar computing devices. Thecomponents shown here, their connections and relationships, and theirfunctions, are meant to be exemplary only, and are not meant to limitimplementations of the inventions described and/or claimed in thisdocument.

Computing device 600 includes a processor 602, memory 604, a storagedevice 606, a high-speed interface 608 connecting to memory 604 andhigh-speed expansion ports 610, and a low speed interface 612 connectingto low speed bus 614 and storage device 606. The processor 602 can be asemiconductor-based processor. The memory 604 can be asemiconductor-based memory. Each of the components 602, 604, 606, 608,610, and 612, are interconnected using various busses, and may bemounted on a common motherboard or in other manners as appropriate. Theprocessor 602 can process instructions for execution within thecomputing device 600, including instructions stored in the memory 604 oron the storage device 606 to display graphical information for a GUI onan external input/output device, such as display 616 coupled to highspeed interface 608. In other implementations, multiple processorsand/or multiple buses may be used, as appropriate, along with multiplememories and types of memory. Also, multiple computing devices 600 maybe connected, with each device providing portions of the necessaryoperations (e.g., as a server bank, a group of blade servers, or amulti-processor system).

The memory 604 stores information within the computing device 600. Inone implementation, the memory 604 is a volatile memory unit or units.In another implementation, the memory 604 is a non-volatile memory unitor units. The memory 604 may also be another form of computer-readablemedium, such as a magnetic or optical disk.

The storage device 606 is capable of providing mass storage for thecomputing device 600. In one implementation, the storage device 606 maybe or contain a computer-readable medium, such as a floppy disk device,a hard disk device, an optical disk device, or a tape device, a flashmemory or other similar solid state memory device, or an array ofdevices, including devices in a storage area network or otherconfigurations. A computer program product can be tangibly embodied inan information carrier. The computer program product may also containinstructions that, when executed, perform one or more methods, such asthose described above. The information carrier is a computer- ormachine-readable medium, such as the memory 604, the storage device 606,or memory on processor 602.

The high speed controller 608 manages bandwidth-intensive operations forthe computing device 600, while the low speed controller 612 manageslower bandwidth-intensive operations. Such allocation of functions isexemplary only. In one implementation, the high-speed controller 608 iscoupled to memory 604, display 616 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 610, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 612 is coupled to storage device 606 and low-speed expansionport 614. The low-speed expansion port, which may include variouscommunication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet)may be coupled to one or more input/output devices, such as a keyboard,a pointing device, a scanner, or a networking device such as a switch orrouter, e.g., through a network adapter.

The computing device 600 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 666, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 624. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 622. Alternatively, components from computing device 600 may becombined with other components in a mobile device (not shown), such asdevice 650. Each of such devices may contain one or more of computingdevice 600, 650, and an entire system may be made up of multiplecomputing devices 600, 650 communicating with each other.

Computing device 650 includes a processor 652, memory 664, aninput/output device such as a display 654, a communication interface666, and a transceiver 668, among other components. The device 650 mayalso be provided with a storage device, such as a microdrive or otherdevice, to provide additional storage. Each of the components 650, 652,664, 654, 666, and 668, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 652 can execute instructions within the computing device650, including instructions stored in the memory 664. The processor maybe implemented as a chipset of chips that include separate and multipleanalog and digital processors. The processor may provide, for example,for coordination of the other components of the device 650, such ascontrol of user interfaces, applications run by device 650, and wirelesscommunication by device 650.

Processor 652 may communicate with a user through control interface 658and display interface 656 coupled to a display 654. The display 654 maybe, for example, a TFT LCD (Thin-Film-Transistor Liquid Crystal Display)or an OLED (Organic Light Emitting Diode) display, or other appropriatedisplay technology. The display interface 656 may comprise appropriatecircuitry for driving the display 654 to present graphical and otherinformation to a user. The control interface 658 may receive commandsfrom a user and convert them for submission to the processor 652. Inaddition, an external interface 662 may be provided in communicationwith processor 652, so as to enable near area communication of device650 with other devices. External interface 662 may provide, for example,for wired communication in some implementations, or for wirelesscommunication in other implementations, and multiple interfaces may alsobe used.

The memory 664 stores information within the computing device 650. Thememory 664 can be implemented as one or more of a computer-readablemedium or media, a volatile memory unit or units, or a non-volatilememory unit or units. Expansion memory 674 may also be provided andconnected to device 650 through expansion interface 672, which mayinclude, for example, a SIMM (Single In Line Memory Module) cardinterface. Such expansion memory 674 may provide extra storage space fordevice 650, or may also store applications or other information fordevice 650. Specifically, expansion memory 674 may include instructionsto carry out or supplement the processes described above, and mayinclude secure information also. Thus, for example, expansion memory 674may be provided as a security module for device 650, and may beprogrammed with instructions that permit secure use of device 650. Inaddition, secure applications may be provided via the SIMM cards, alongwith additional information, such as placing identifying information onthe SIMM card in a non-hackable manner.

The memory may include, for example, flash memory and/or NVRAM memory,as discussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 664, expansionmemory 674, or memory on processor 652, that may be received, forexample, over transceiver 668 or external interface 662.

Device 650 may communicate wirelessly through communication interface666, which may include digital signal processing circuitry wherenecessary. Communication interface 666 may provide for communicationsunder various modes or protocols, such as GSM voice calls, SMS, EMS, orMMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA600, or GPRS, among others.Such communication may occur, for example, through radio-frequencytransceiver 668. In addition, short-range communication may occur, suchas using a Bluetooth, WiFi, or other such transceiver (not shown). Inaddition, GPS (Global Positioning System) receiver module 670 mayprovide additional navigation- and location-related wireless data todevice 650, which may be used as appropriate by applications running ondevice 650.

Device 650 may also communicate audibly using audio codec 660, which mayreceive spoken information from a user and convert it to usable digitalinformation. Audio codec 660 may likewise generate audible sound for auser, such as through a speaker, e.g., in a handset of device 650. Suchsound may include sound from voice telephone calls, may include recordedsound (e.g., voice messages, music files, etc.) and may also includesound generated by applications operating on device 650.

The computing device 650 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 680. It may also be implemented as part of asmartphone 682, personal digital assistant, or other similar mobiledevice.

Various implementations of the systems and techniques described hereincan be realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. As used herein, the terms “machine-readable medium”“computer-readable medium” refers to any computer program product,apparatus and/or device (e.g., magnetic discs, optical disks, memory,Programmable Logic Devices (PLDs)) used to provide machine instructionsand/or data to a programmable processor, including a machine-readablemedium that receives machine instructions as a machine-readable signal.The term “machine-readable signal” refers to any signal used to providemachine instructions and/or data to a programmable processor.

To provide for interaction with a user, the systems and techniquesdescribed herein can be implemented on a computer having a displaydevice (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display)monitor) for displaying information to the user and a keyboard and apointing device (e.g., a mouse or a trackball) by which the user canprovide input to the computer. Other kinds of devices can be used toprovide for interaction with a user as well; for example, feedbackprovided to the user can be any form of sensory feedback (e.g., visualfeedback, auditory feedback, or tactile feedback); and input from theuser can be received in any form, including acoustic, speech, or tactileinput.

The systems and techniques described herein can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed herein), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

A number of embodiments have been described. Nevertheless, it will beunderstood that various modifications may be made without departing fromthe spirit and scope of the invention.

In addition, the logic flows depicted in the figures do not require theparticular order shown, or sequential order, to achieve desirableresults. In addition, other steps may be provided, or steps may beeliminated, from the described flows, and other components may be addedto, or removed from, the described systems.

1. A computer-implemented method, comprising: executing, by a processorof a user device, content on the user device; detecting, by the userdevice, at least one external device within a set range of the userdevice; identifying the at least one external device for output of thecontent being executed by the user device based on output capabilitiesof the at least one external device and output characteristics of thecontent being executed by the user device; and transferring execution ofthe content from the user device to the at least one external device foroutput by the at least one external device.
 2. The computer-implementedmethod of claim 1, wherein detecting the at least one external devicewithin the set range of the user device includes detecting the at leastone external device within a field of view of the user device.
 3. Thecomputer-implemented method of claim 1, wherein identifying the at leastone external device includes: detecting physical features in imageinformation captured by the user device; identifying a location of theuser device based on the detected physical features; and identifying theat least one external device based on the identified location of theuser device.
 4. The computer-implemented method of claim 3, wherein theuser device includes a handheld device and a head mounted displaydevice, and wherein executing content includes executing content on thehandheld device; and detecting physical features in the imageinformation includes detecting physical features in image informationcaptured within a field of view of the head mounted display device. 5.The computer-implemented method of claim 3, wherein identifying thelocation of the user device based on the detected physical featuresincludes identifying the location of the user device based on acomparison of previously stored features in a database and accessible tothe user device via a server.
 6. The computer-implemented method ofclaim 1, wherein detecting the at least one external device within theset range of the user device includes at least one of: detecting the atleast one external device as a previously connected device; detectingthe at least one external device based on an electromagnetic signal;detecting the at least one external device within a shared network; ordetecting the at least one external device within a network ofpreviously connected devices.
 7. The computer-implemented method ofclaim 1, wherein detecting the at least one external device within theset range of the user device includes detecting the at least oneexternal device within a field of view of a user of the user device. 8.The computer-implemented method of claim 1, wherein transferring theexecution of the content from the user device to the at least oneexternal device for output by the at least one external device includes:outputting a request for user verification prior to transferring thecontent from the user device to the at least one external device;receiving a user verification input verifying the transfer of contentfrom the user device to the at least one external device; andtransferring the execution of the content from the user device to the atleast one external device in response to receiving the user verificationinput.
 9. The computer-implemented method of claim 8, wherein outputtingthe request for user verification includes outputting at least one of avisual request or an audible request, and receiving the userverification input includes receiving at least one of a touch input oran audible input.
 10. A system, including: an output system including anaudio output device and an image output device; a sensing system; atleast one processor; and a memory storing instructions that, whenexecuted by the at least one processor, cause the at least one processorto: output, by the output system, content executed by the at least oneprocessor; detect, by the sensing system, at least one external devicewithin a set range of the sensing system; identify the at least oneexternal device for output of the content being executed by the userdevice based on output capabilities of the at least one external deviceand output characteristics of the content being output by the outputsystem; and transfer execution of the content to the at least oneexternal device for output by the at least one external device.
 11. Thesystem of claim 10, wherein, in detecting the at least one externaldevice the instructions cause the at least one processor to at least oneof: detect the at least one external device based on image informationcaptured by the sensing system; detect the at least one external deviceas a previously connected device; detect the at least one externaldevice based on an electromagnetic signal; detect the at least oneexternal device within a shared network; or detect the at least oneexternal device within a network of previously connected devices. 12.The system of claim 11, wherein, in detecting the at least one externaldevice based on image information captured by the sensing system, theinstruction cause the at least one processor to: detect physicalfeatures in the image information; identify a location based on acomparison the detected physical features and with previously storedfeatures in a database accessible to the system via a server; andidentify the at least one external device based on the identifiedlocation.
 13. The system of claim 10, wherein, in transferring theexecution of the content to the at least one external device, theinstructions cause the at least one processor to: output a request foruser verification prior to transferring the content to the at least oneexternal device; receive a user verification input verifying thetransfer of content to the at least one external device; and transferthe execution of the content to the at least one external device inresponse to receiving the user verification input.
 14. The system ofclaim 13, wherein the request for user verification includes at leastone of a visual request or an audible request, and the user verificationinput includes at least one of a touch input or an audible input. 15.The system of claim 10, wherein the output system, the sensing system,the at least one processor and the memory are installed in at least oneof a handheld electronic device or a head mounted display device thatare operable within a network.